Improved 3D Model Search for Facial Feature Location and Pose Estimation in 2D images

This paper tackles the problem of accurately matching a 3D deformable face model to images in challenging real-world scenarios with large amounts of head movement, occlusion, and difficult lighting conditions (Figure 2). A baseline system [1] based on the ASM [2] involves searching with a set of view-dependent local patches to locate image features and using their positions to update the face shape model parameters (Figure 1). We show here two modifications that lead to improvements in performance and can be applied in other similar systems. These are: a simple method for weighting the relative importance of each located match for model fit; and explicitly searching for occluding boundaries, which prevents the model from shrinking or rotating rather than changing shape. We demonstrate the improvements on both standard test sets and on a series of difficult in-car driver videos.